1. Field of the Invention
The invention relates generally to the area of non-manual human control of external systems. More specifically, the invention provides a control system with which a user may control an external device by regulating control signals in response to a collective manifestation of electroencephalographic and electromyographic biopotentials. The invention further provides the capability for the user to play music and games and create visual art as training aids for improving system control and for entertainment and relaxation.
2. Summary of the Related Art
Through the years there has been significant research in the area of detecting and observing various electric potentials generated within the human body for medical diagnosis, biofeedback control of mental and physical states, and control of external devices. In that work, it is well-known to detect on the outer surface of the head electroencephalographic ("EEG") biopotentials or brainwaves which demonstrate continuous electrical activity in the brain. The intensities of the brain waves or EEG on the surface of the scalp range from zero to 300 microvolts, and their frequencies range from once every few seconds to 50 or more per second. Much of the time, the brain waves are irregular, and no general pattern can be discerned in the EEG. However, at other times distinct patterns are present. For classification purposes, the EEG has been divided into a number of frequency spectrums. These frequency spectrums can be classified into `alpha` (8 Hz to 13 Hz), `beta` (14 Hz to 50 Hz), `theta` (4 Hz to 7 Hz), and `delta` (below 3.5 Hz). Activities within the various EEG spectrums have been correlated to states of sleep, relaxation, active thought, etc. Depending on :he nature of the activity of interest, it is well-known to detect EEG waves at different areas on the scalp as a function of the part of the brain of interest.
By providing a feedback: of EEG biopotentials in a particular EEG spectrum, a subject may be trained to emphasize or de-emphasize an activity associated with that EEG spectrum thereby reinforcing or diminishing the mental and physical state associated therewith. Further, work has been done with a subject to provide a feedback of EEG activity in a particular spectrum, for example, the alpha spectrum of 8 Hz-13 Hz. Using that feedback, the subject learns to control the magnitude of the alpha spectrum to energize a switch or other external device. In other work, through training, a subject is able to generate an alpha biopotential in response to an external stimulus.
A disadvantage in all of the above work is that one measurement site produces only one control. Using multiple bandpass filters or a Fast Fourier Transform algorithm (FFT), the EEG is divided into a number of frequency spectrums. By employing these techniques, users have been able to work with the time varying EEG spectrum magnitudes. While pure EEG signals may be divided into a number of frequency spectra correlated to mental states, it is very difficult to learn to control those spectra and mental states and to maintain such control over time without extensive practice.
It has been suggested that training time can be reduced in the alpha band by phase matching the biofeedback signal to the bandpassed alpha spectrum signal. This is accomplished by delaying the biofeedback signal by one complete cycle. The delay is set as a function of the predetermined dominant alpha peak frequency of the subject. This approach requires that each subject have a predominant alpha peak frequency that can be measured before training. However, one problem is that not all subjects produce spontaneous alpha. A further disadvantage is that this form of phase loop closure will only work for alpha control because theta and beta dominant peaks are not easily predetermined. It also assumes that the dominant alpha peak frequency of the subject wants to be kept constant over a session.
The time varying characteristics of a bandpass filter output can be used to create an estimate of phase information. Likewise the FFT can provide phase measures as well as magnitude measures. Thus phase information can be used as a feedback signal as well as magnitude. However, other than the attempt to create phase matching to an alpha peak frequency as discussed above, there are no instances in the prior art in which use of phase information is successfully incorporated into a biofeedback paradigm.
The contraction of skeletal muscle is preceded by a sequence of rapid changes in the muscle nerve fiber membrane potential. This sequence of potential changes is called an action potential. Each time an action potential passes along a muscle fiber a small portion of the electrical current spreads away from the muscle as far as the surface of the skin. If many muscle fibers contract simultaneously, the summated electrical potentials at the skin may be great. These summated electrical potentials are referred to as electromyographic biopotentials (EMG).
EMG biopotentials have also been detected and used for various forms of medical diagnosis and biofeedback control. Strong EMG biopotentials are usually considered to occur in a range of approximately 100 Hz-3000 Hz; but since the EMG is the summation of numerous action potentials, EMG biopotentials will occur below 100 Hz as well. Therefore, EMG biopotentials contain frequency components between zero and 100 Hz. EMG biopotentials are typically detected at the site of muscle activity, for example, at the jaw to monitor jaw tension or around the eyes to detect ocular muscle activity. EMG biopotentials may be detected for medical diagnostic purposes in which a patient observes their own muscle tension as a biofeedback signal. In addition, EMG biopotentials may be detected for the purposes of activating a switch mechanism to control an external device. Even though EMG biopotentials are somewhat easier to control because they are produced by a physical activity, any use in the prior art work of EMG signals is in response to an averaged magnitude over a spectrum centered at 100 Hz or more. That averaged magnitude is used to control a single activity or switch. Therefore, a limitation of traditional EMG signal processing is only a single channel of control.
Most of the prior work makes extraordinary efforts to work with signals representing either pure EEG biopotentials or pure EMG biopotentials. In the examples of EEG work, the detection and processing of EEG biopotentials in the range of approximately 0.5 Hz-35 Hz includes processing to reject EEG when it contains artifacts of EMG biopotentials. One approach is to inhibit the production of the feedback signal if an undesirable attribute appears in the EEG biopotentials. Another approach is to obtain a multiplicity of EEG and EMG signals and inhibit feedback when any of the EMG signals exhibit undesirable characteristics. There is a potential problem in using an inhibit approach to deal with an artifact. If a subject simultaneously produces the correct EEG response while producing an inappropriate EMG response, inhibition provides an ambiguous feedback cue. In that case, the absence of feedback due to inhibition suggests to the subject that they are not producing the appropriate EEG response when in fact they are.
Other approaches that attempt to deal with artifacts include: providing subjects with a cross-hair fixation point to limit eye movements, making EEG measurements as far away from potential EMG sources as possible, for example, the occipital and parietal regions of the scalp, and the sensing of and subtraction of the corneoretinal potential from the EEG. All of these approaches have inherent disadvantages. They either provide ambiguous or false feedback cues, require a multiplicity of measurement sites, or they reject the potential usefulness that might be gained by the simultaneous presence of both the EEG and EMG biopotentials. Therefore, even though there has been significant work with EEG and EMG biopotentials for several decades, there have been few practical results.